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Abstract 

As  OSD  seeks  to  field  new  capabilities  while  working  to  reduce  cost  and  risk,  it 
becomes  imperative  that  systems  engineering  tools  evolve.  Traditionally,  cost/schedule 
monitoring,  technology  assessment,  and  performance  analyses  have  been  conducted  as 
independent  activities  focused  on  systems.  However,  as  systems  become  more  complex 
and  entwined  into  operating  as  components  of  System  of  Systems  (SoS),  the  need  for  more 
insight  during  the  design  and  development  stages  increases.  This  dictates  the  need  for  a 
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methodology  for  SoSs  that  allows  for  fully  integrated  analysis  and  trade-offs  of  the  technical, 
cost,  and  schedule  design  spaces. 

The  US  Navy  (PMS  420,  SSC  Pacific),  Northrop  Grumman,  and  the  Stevens  Institute 
of  Technology  are  collaborating  to  develop  such  a  comprehensive  financial  and  portfolio 
management  methodology  for  SoSs.  The  concept  leverages  the  System  Readiness  Level 
(SRL)  as  a  measure  of  SoS  development  and  coalesces  a  system  performance  monitoring 
approach  that  provides  insight  into  both  current  and  anticipated  performance.  Additionally,  a 
methodology  for  understanding  the  impact  of  technical  trades  within  a  SoS  is  introduced. 
Together,  these  tools  allow  for  a  true  trade-off  analysis  capability  that  can  be  used  to 
examine  the  extent  to  which  a  set  of  technology  options  either  meet  budget  constraints  or 
maximize  performance. 

The  Challenges  of  System  of  Systems  Management 

The  Department  of  Defense  (DoD)  has  seen  a  growth  in  the  acquisition  of  SoSs  over 
the  last  few  decades.  This  trend  is  expected  to  continue  as  the  DoD  increases  focus  on 
capabilities  without  changing  its  system  oriented  acquisition  organization.  While  providing 
significant  opportunities  for  extending  mission  capabilities  through  the  integration  of  existing 
and  new  capabilities  into  a  synergistic  SoS,  there  exists  significant  systems  engineering 
challenges  related  to  the  integration  and  management  of  SoS.  These  engineering 
challenges  are  discussed  in  the  Systems  Engineering  Guide  for  Systems  of  Systems 
(ODUSD(A&T)SSE,  2008). 

One  example  of  the  challenge  presently  facing  SoS  Program  Managers  (PMs)  is  the 
understanding  of  the  SoS  technical  maturity.  Historically,  the  Technology  Readiness  Level 
(TRL)  methodology  has  been  a  key  gauge  of  the  technical  maturity  for  individual  systems 
within  the  DoD  for  the  better  part  of  two  decades.  However,  when  TRL  is  applied  to 
components  within  a  SoS,  the  model  of  using  individual  technology  maturity  as  a  measure  of 
readiness  for  SoS  development  quickly  breaks  down.  TRLs  simply  do  not  account  for 
integration  maturity  or  the  complexity  of  bringing  together  any  number  of  independent 
technologies  to  function  as  a  SoS. 

Similar  problems  also  become  apparent  with  many  other  systems  engineering  and 
program  management  tools  when  applied  in  a  SoS  context,  including  Technical 
Performance  Measures  (TPMs)  as  used  to  track  progress  toward  achieving  Key 
Performance  Parameters  (KPPs)  and  Earned  Value  (EV)  Management  (to  track 
cost/schedule).  Existing  tools  simply  do  not  provide  sufficient  insight  into  SoS  development, 
contributing  to  a  rash  of  complex  development  and  acquisition  projects  that  have  gone 
astray.  In  a  2006  study  (GAO,  2006,  September  14)  the  Government  Accountability  Office 
(GAO)  noted  that  a  lack  of  insight  into  the  technical  maturity  of  complex  systems  during 
development  has  contributed  to  an  environment  of  significant  cost  overruns,  schedules  slips 
leading  to  program  delays,  canceled  acquisition  efforts,  and  reduced  system  performance  at 
fielding.  In  case  after  case,  failure  is  most  commonly  not  found  at  the  technology 
development  level,  but  rather  at  the  point  of  a  combination  of  two  or  more  elements. 

This  paper  provides  insight  to  the  methodologies  and  tools  being  developed  and 
used  by  the  Littoral  Combat  Ship  (LCS)  Mission  Modules  Program  Office  (PMS  420)  to 
assist  the  PM  and  his  staff  in  meeting  the  SoS  development  challenge. 
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The  Mission  Modules  Program — An  Acknowledged  SoS 
Example 

Acknowledged  System  of  Systems  Definition 

The  DoD  System  of  Systems  Engineering  Guide  defines  an  acknowledged  SoS  as  a 
set  or  arrangement  of  systems  that  results  when  independent  and  useful  systems  are 
integrated  into  a  larger  system  that  delivers  unique  capabilities.  An  acknowledged  SoS  has 
recognized  objectives,  a  designated  manager,  and  resources.  Within  an  acknowledged  SoS, 
the  constituent  Mission  Systems  (MS)  retain  their  independent  ownership,  objectives, 
funding,  development  and  sustainment  approaches.  Changes  in  the  MSs  are  based  on 
collaboration  between  the  SoS  PM  and  the  MS  PM.  This  complicates  the  task  of  a  SoS  PM 
and  system  engineer  who  must  navigate  the  evolving  plans  and  development  priorities  of 
the  SoS  constituent  systems,  along  with  their  asynchronous  development  schedules,  to  plan 
and  orchestrate  evolution  of  the  SoS  toward  SoS  objectives. 

The  LCS  Mission  Modules  Program  as  an  Acknowledged  System  of  Systems 

The  LCS  Mission  Modules  Program  Office  (PMS  420)  was  established  by  the 
Assistant  Secretary  of  the  Navy  for  Research,  Development,  and  Acquisition  (ASN  RD&A) 
on  October  1,  2003  within  the  Program  Executive  Office-Littoral  and  Mine  Warfare  (PEO 
LMW)  for  the  development,  acquisition,  and  sustainment  of  the  modular  Mission  Packages 
(MPs).  The  initial  focus  of  PMS  420  was  to  take  existing  independent  capabilities  in  the 
fields  of  surface  warfare  (SUW),  mine  countermeasure  (MCM),  and  anti-submarine  warfare 
(ASW)  and  to  integrate  and  modularize  those  capabilities  to  provide  deployable  and 
swappable  warfighting  capabilities  for  the  LCS.  Thus,  the  LCS  MPs  meet  the  definition  of 
being  a  SoS  as  they  are  made  up  of  individual  MSs,  including  vehicle,  communication, 
sensor,  or  weapon  systems;  support  equipment,  including  support  containers,  or  vehicle 
cradles;  software;  mission  crew  detachments;  and  aviation  systems;  these  are  then 
integrated  into  a  larger  system  to  deliver  unique  capability.  As  the  charter  of  PMS  420  is  to 
acquire,  integrate,  modularize,  and  sustain  focused  warfighting  capabilities  from  existing 
program  lines,  PMS  420  primarily  serves  as  a  Ships  Acquisition  Program  Manager  (SHAPM) 
with  a  focus  on  acquiring  the  individual  mission  systems  from  Participating  Acquisition 
Resource  Managers  (PARMS)  who  manage  existing  product  lines  and  programs  of  records. 
This  lack  of  direct  management  responsibility  for  the  individual  mission  systems  means  that 
the  SoSs  comprising  the  MPs  are  an  acknowledged  SoS. 

Mission  Package  Explained 

The  hierarchal  MP  concept,  illustrated  in  Figure  1,  is  best  described  in  three  layers. 

These  layers  are: 

Mission  System  (MS)  =  a  single  Vehicle,  Communication,  Sensor,  or  Weapon 
System 

Mission  Module  (MM)  =  a  combination  of  mission  systems  +  Support 
Equipment  +  Software  that  provide  a  unique  mission  capability 

Mission  Package  (MP)  =  the  collection  of  MMs  +  Mission  Crew  Detachments 
+  Support  Aircraft  that  provides  the  required  ability  to  conduct  a  focused 
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warfighting  mission  as  required  by  the  LCS  Capability  Development 
Document  (CDD). 

A  MP  consists  of  MSs  integrated  into  warfighting  responsive  MMs,  mission  crew 
detachment  and  mission  configured  aircraft  with  their  composite  aviation  crew  detachment. 
MMs  combine  MSs  (vehicles,  sensors,  weapons)  and  support  equipment  that  install  into  the 
Seaframe  via  standard  interfaces. 
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MPAS  -  Mission  Package  Application  Software 
MPCE  -  Mission  Package  Computing  Environment 
MVCS  -Multi  Vehicle  Communication  System 


Figure  1.  Mission  Package  Defined 

MSs  are  sized  to  fit  inside  standard  ten-  or  twenty-foot  International  Organization  for 
Standardization  (ISO)  support  containers  (SCs),  or  on  ISO  compliant  flat  racks  and  vehicle 
cradles.  Using  ISO  SCs  simplifies  shipping,  storage,  availability  of  correct  handling 
equipment,  and  container  movement  from  shore  to  ship  and  ship  to  shore  as  the  Navy  is 
able  to  leverage  the  intermodal  transportation  resources  used  for  shipping  commercial  cargo 
worldwide.  MP  reconfiguration  will  occur  in  homeport  or  overseas,  using  pre-positioned  MPs 
or  MPs  that  have  been  transported  into  theater  by  air  or  sea  and  staged  near  the  LCS 
operating  area. 

MPs  can  be  swapped  in  order  to  reconfigure  the  ship  for  a  different  mission  in  a  short 
period  of  time,  giving  a  Combatant  Commander  a  uniquely  flexible  response  to  changing 
warfighting  requirements.  To  achieve  this  flexibility,  the  Navy  is  developing  and  procuring 
specific  numbers  of  MPs  to  meet  the  Fleet’s  warfighting  requirements.  The  quantity  of  each 
MP  type  differs  based  on  analysis  of  projected  operational  needs. 

Mission  Package  Status 

The  first  two  LCS  ships,  USS  Freedom  (LCS-1)  and  USS  Independence  (LCS-2), 
have  been  delivered  to  the  Navy  along  with  prototype  MPs  to  the  Navy  for  use  in  testing  the 
designs  and  the  concept  of  modular  warfighting  capability.  The  LCS  MM  Program  (hereafter 
referred  to  as  “the  program”)  is  presently  proceeding  to  a  Milestone  B  acquisition  decision. 
The  first  three  MPs  (1  each  of  SUW,  ASW,  and  MCM)  have  completed  Design  Readiness 
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Review  (DRR)  and  are  engaged  in  further  developmental  testing  of  their  capabilities  either 
independently  in  End-to-End  (E2E)  testing  and/or  by  being  integrated  onto  the  LCS  and 
tested  as  part  of  the  Seaframes  integrated  warfighting  capability.  The  rollout  for  the  first 
MCM  MP  was  on  September  14,  2007.  Phases  1  and  2  of  E2E  testing  for  MCM  MP  1  were 
completed  September  25,  2008,  and  August  19,  2009,  respectively.  Phase  3  of  E2E  testing 
for  MCM  MP  1  is  scheduled  for  Summer  201 0.  The  first  ASW  MP  was  rolled  out  on 
September  19,  2008.  E2E  testing  for  ASW  MP  1  was  completed  April  3,  2009.  ASW  MP  1 
Developmental  Test  (DT)  is  scheduled  for  Summer  2010.  The  first  SUW  MP  was  rolled  out 
on  July  1 1 ,  2008.  E2E  testing  for  SUW  MP  1  was  completed  in  July  2009.  The  first  SUW  MP 
was  recently  called  on  for  an  early  deployment  mission.  In  February  2010,  LCS  1  deployed 
with  the  first  SUW  MP  augmented  with  a  prototype  Maritime  Security  Module  (MSM)  to 
provide  Visit,  Board,  Search,  and  Seizure  (VBSS)  capability  to  the  Southern  Command. 

PMS  420  Progress  on  Addressing  the  Management  Challenges  for  SoS 

Programs 

PMS  420,  since  its  founding,  has  recognized  the  challenge  of  leading  an 
acknowledged  SoS  development  and  quickly  began  development  of  novel  system 
engineering  tools  and  methodologies,  designed  to  ultimately  reduce  risk  and  provide 
enhanced  management  (technical,  cost,  schedule)  insight  into  the  SoS  problem.  Four  initial 
areas  of  traditional  programmatic  concern  have  been  developed  to  date.  Lessons  learned  by 
PMS  420,  approaches  used  and  their  benefits  will  briefly  be  discussed  within  this  paper. 
These  tools/processes  include  the  areas  outlined  below. 

1 .  SRL  Used  to  Determine  SoS  Maturity  Analysis:  The  System  Maturity  Model 
(SMM),  developed  by  the  Stevens  Institute  of  Technology  (SIT)  and  Northrop 
Grumman  underfunding  provided  by  PMS  420.  The  SMM  has  been  applied 
for  the  purpose  of  monitoring  the  maturity  and  integration  status  of  individual 
technologies  within  the  MP  SoSs  for  PMS  420  and  will  be  discussed  in 
Section  3.1 . 

2.  Requirements  Management  &  the  Drive  towards  Commonality  in  a  SoS 
World:  Requirements  management  is  a  significant  challenge  within  an 
individual  system  development  effort.  This  becomes  even  more  challenging 
within  the  interrelated  development  environment  of  the  three  SoSs  that  PMS 
420  faced.  As  an  acquiring  PM,  PMS  420  has  limited  ability  to  impact  Life 
Cycle  Cost  (LCC)  of  acquired  capabilities.  However,  through  implementation 
of  an  effective  centralized  and  consolidated  requirements  management 
capability,  PMS  420  is  using  commonality  as  a  means  to  drive  down  cost. 

This  approach  will  be  expanded  upon  in  Section  3.2. 

3.  Expanding  Financial  Management  past  EV:  Financial  and  task  management 
within  a  SoS  is  a  complex  task.  PMS  420  required  new  processes  and  tools 
that  could  improve  the  PM’s  ability  to  monitor  and  review  task  execution  and 
earned  value,  support  multi-year  pre-planning  of  research,  development, 
procurement,  and  sustainment  efforts  at  warfare  centers  and  contractors.  In 
addition,  there  was  a  need  for  greater  insight  into  the  cost  of  risk 
management  activities  and  a  desire  to  reduce  funding  document  touch-time 
and  rejections  rates.  PMS  420  has  developed  such  a  tool,  and  a  description 
of  the  tool  and  approach  used  by  PMS  420  to  accomplish  this  will  be 
expanded  upon  in  Section  3.3 
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4.  Understanding  and  Influencing  SoS  Reliability:  Traditionally,  system  reliability 
has  been  determined  through  the  calculation  of  independent  critical 
component  reliability  in  the  system  and  defined  by  the  value  of  Operational 
Availability  (A0).  In  a  SoS,  especially  in  a  mission-focused  area,  this  approach 
may  no  longer  be  the  best  metric  for  use.  PMS  420  has  been  evaluating  an 
approach  based  on  mission  completion  and  the  use  of  reliability  block 
diagrams  developed  to  represent  mission  strings,  which  will  be  expanded 
upon  in  Section  3.4. 

In  addition  to  the  tools  that  have  been  developed  to  date,  two  additional  areas  will  be 
presented  that  PMS  420  is  presently  investigating  that  are  designed  to  further  enhance  the 
PM’s  ability  to  gain  insight  into  understanding  the  status  and  risks  associated  with  SoS 
development.  These  tools,  while  still  under  development  and  evaluation  by  PMS  420,  are 
designed  to  assist  the  SoS  PM  in  the  areas  of  conducting  capability  tradeoffs  and  in  the 
understanding  of  the  ability  to  achieve  required  performance.  Specifically,  these  tools  are 
designed  to  provide  insight  into  areas  listed  below. 

1 .  Evaluating  the  Impact  of  Technology  Insertion:  One  of  the  strengths 
envisioned  from  the  MP  SoS  is  their  perceived  ability  to  adapt  to  and  rapidly 
and  effectively  incorporate  new  technologies  that  can  provide  increased 
warfighting  capabilities.  If  the  integration  and  maturation  risks  are  not  fully 
understood,  a  perceived  improvement  could  actually  lead  to  a  decreased 
capability  or  significantly  increased  programmatic  costs.  To  avoid  these 
negative  effects,  PMS  420  has  been  developing  a  methodology  to  assess  the 
impact  of  technology  insertions  in  support  of  conducting  tradeoff  analysis. 
This  proposed  approach  will  be  expanded  upon  in  section  4.1. 

2.  Predicting  SoS  Performance:  One  of  the  challenges  of  an  acknowledged 
SoS,  is  that  the  SoS  PM  may  have  little  ability  to  monitor  performance 
development  of  the  individual  systems.  This  issue  is  further  complicated  by 
the  fact  that  individual  system  performance,  when  integrated  into  a  SoS,  may 
be  different.  How,  then,  can  the  SoS  PM  determine  if  they  are  on  track  to 
satisfy  the  KPPs  for  the  SoS?  PMS  420  has  been  developing  a  Performance 
Level  Monitoring  Model  that  seeks  to  provide  the  PM  with  this  form  of  insight. 
The  proposed  approach  will  be  expanded  upon  in  section  4.3. 

Systems  Engineering  Management  and  Insight  in  SoS 
Programs 

As  discussed  in  the  previous  sections  of  this  paper,  a  SoS  usually  does  not  directly 
control  the  development  of  the  majority  of  the  technologies  comprising  the  acknowledged 
SoS.  This  is  a  common  situation  and  poses  significant  challenges  to  management  in 
understanding  where  the  end  capability  of  a  System  stands  with  respect  to  providing  the 
required  level  of  performance  as  specified  in  the  KPPs  and  the  TPMs  and  in  understanding 
the  level  of  developmental  and  integration  risk  of  the  individual  technologies  composing  the 
SoS.  To  resolve  this  area  of  engineering  management  concern,  PMS  420  started  the 
development  of  a  portfolio  of  SoS  Management  tools.  This  section  of  the  paper  discusses 
some  of  these  tools  and  lays  the  foundation  for  the  later  discussion  of  future  efforts. 
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SRL  Used  to  Determine  SoS  Maturity  Analysis 

LCS  MPs  will  deliver  required  capability  via  the  fielding  of  a  series  of  incremental 
MPs  until  full  capability  that  satisfies  the  LCS  CDD  is  reached.  For  example,  Increment  1  of 
the  MCM  MP  provides  capability  for  the  detection  and  neutralization  of  volume  and  bottom 
mines.  Increment  2  of  the  MCM  MP  introduces  inshore  detection  capability  via  the  Coastal 
Battlefield  Reconnaissance  and  Analysis  (COBRA)  system.  Increment  3  introduces 
additional  MCM  capability  to  the  Fleet,  including  a  magnetic  and  acoustic  sweep  capability 
to  address  the  bottom/buried  mines  threat  using  the  AN/ALQ-220  Organic  Airborne  and 
Surface  Influence  Sweep  (OASIS)  system.  The  full  MCM  baseline  capability  will  be  achieved 
by  Increment  4  in  FY17,  which  introduces  the  AN/AWS-2  Rapid  Airborne  Mine  Clearance 
System  (RAMICS),  that  neutralizes  near  surface  and  floating  mines. 

As  presented  to  this  symposium  last  year,  in  order  to  gain  insight  and  manage  the 
development  maturity  of  these  incremental  deliveries,  PMS  420  has  implemented  an 
emerging  concept  known  as  the  SRL  (Forbes,  Volkert,  Gentile  &  Michaud,  2009).  By  pairing 
the  traditional  TRL  scale  with  a  new  series  of  criteria  known  as  the  Integration  Readiness 
Level  (IRL),  a  more  complete  look  at  true  system  maturity  can  be  obtained  (Sauser, 
Ramirez-Marquez,  Magnaye  &  Tan,  2008).  Linder  this  methodology,  the  readiness  of  each 
technology  is  still  considered,  but  instead  of  being  a  stand-alone  metric  for  determining 
readiness  for  incorporation,  it  is  analyzed  in  concert  with  both  its  integration  requirements 
and  the  maturity  of  other  technologies  with  which  it  interfaces.  The  calculation  of  SRL  is 
described  in  the  above-referenced  papers.  The  SRL  methodology  has  been  highly 
successful  on  the  program  and  has  paid  dividends  in  terms  of  both  increasing  decision¬ 
maker  visibility  into  true  system  status  and  allowing  for  pre-emptive  actions  to  be  taken  to 
mitigate  potential  developmental  issues. 

An  example  of  the  use  of  this  approach  by  PMS  420  in  understanding  the  impact  to 
the  program  of  technology  options  is  shown  in  Figure  2,  where  the  SRL  for  the  initial 
configuration  (on  the  left  side  of  the  figure)  of  the  MCM  MP  number  1  is  calculated  as  0.57. 

In  this  configuration,  one  system  component,  the  Multi-Vehicle  Communications  System  for 
the  Remote  Multi-Mission  Vehicle  (MVCS  (RMMV)),  is  early  in  its  maturity  and  is  lagging 
most  other  components,  both  in  its  technology  and  its  integration  readiness.  This 
configuration  resulted  in  a  lower  than  acceptable  overall  SRL  of  0.57,  beyond  the  risk 
threshold  of  the  program. 

The  program  evaluated  the  replacement  of  this  lagging  component  with  the 
combination  of  a  Data  Link  System  (DLS)  both  on-board  and  on  the  RMMV,  each  of  which 
have  both  better  TRLs  and  IRLs.  This  is  shown  on  the  right  side  of  Figure  2.  In  this  manner, 
the  overall  SRL  of  the  MCM  MP  1  increased  to  0.67,  now  within  the  range  acceptable  to  the 
established  risk  threshold  of  the  program. 

The  program  has  used  this  methodology  to  monitor  developmental  status  by 
incorporating  it  into  a  continuing  quarterly  evaluation  of  the  SRL  level  for  each  of  the  mission 
packages.  This  consistent  evaluation  allows  the  PMS  420  PM  to  better  understand 
maturation  of  the  individual  MP  SoS  and  of  each  increment  within  the  SoS.  In  turn,  this 
provides  him  with  a  greater  understanding  of  the  program’s  technical  status,  enabling  the 
PM  to  better  maintain  and  manage  the  development  risk  of  the  MPs  as  they  progress 
through  design  and  development. 
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Figure  2.  Initial  and  Enhanced  Readiness  of  MCM  MP  1 

An  example  of  the  progress  of  the  SRL  level  for  two  increments  of  the  SUW  MPs  is 
illustrated  in  Figure  3  for  the  SUW  MP.  The  graph  demonstrates  how  the  SRL  value  of 
different  increments  can  be  affected  by  both  similar  and  [different]  systems  that  make  up  the 
increments.  The  main  dip  in  the  graph  (4-5)  indicates  a  problem  that  was  identified  with  a 
shared  system  between  the  increments  discovered  during  interface  testing,  while  the  rise  in 
the  graph  (5-6)  demonstrates  the  results  of  correcting  that  identified  problem.  The  lower 
overall  SRL  of  the  SUW  Increment  2  is  associated  with  the  lower  maturity  level  of  one  its 
component  technologies,  which  is  being  consistently  monitored  by  PMS  420. 


- SUW  Increment  1 

SUW  Increment  2 


Figure  3.  SRL  Values  Over  Time 


Since  the  initial  presentation  of  the  SRL  method,  the  program  has  developed  and 
documented  a  comprehensive  process  for  System  Maturity  Assessments  (SMA)  and 
described  its  application  to  generic  SoSs.  The  SMA  process  is  iterative  with  a  structured  set 
of  well-developed  tasks  that  are  described  in  detail  in  the  System  Maturity  Assessment 
Guide  (PMS  420,  2009)  and  illustrated  in  Figure  4.  The  first  three  steps  of  this  process  need 
only  be  conducted  during  initial  system  architecture  development.  Once  the  system 
architecture  and  subsequent  system  designs  have  been  placed  under  configuration  control, 
successive  assessment  iterations  need  only  review  the  previous  TRL  and  IRL  criteria  for  any 
updates  due  to  development  progress  and  then  recalculate  the  SRL  with  updates  to 
reporting  mechanisms  conducted  as  needed.  The  fundamental  basis  of  the  SMA  process  is 
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the  proper  creation  of  an  assessment  framework  to  include  technologies,  integrations,  and 
their  resulting  architecture.  It  is  also  imperative  that  buy-in  from  all  stakeholders  be  obtained 
in  order  to  ensure  common  understanding  among  all  participants  with  regard  to  both  what 
will  be  evaluated  and  in  what  manner. 


1.  Develop  System  Architectures 


2.  Determine  Criticality 


3.  Build  Assessment  Process 

•  Customize  applicable  TRL/  IRL  criteria 

•  Build  SRL  advancement  schedule 

•  Tie  criteria  to  program  test  events  / 
milestones 

•  Review  proposed  criteria,  schedule,  and 
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•  Approve  assessment  framework 


Architectures  and  framework  are  locked  after  approval  and  will  remain  so  unless  the  program  is  re-baselined 


4.  Conduct  System  Maturity  Analysis  w/  SRL 


5.  Interpret  and  Apply  Results 
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Outputs  of  the  analysis  are  analyzed  against  projected  cost 
and  schedule  data  to  determine  current  development  status 

Future  planning  can  also  be  conducted  through  trade-off 
analyses  and  risk  management  activities 


Figure  4.  SMA  Development  Process  Flow 
Requirements  Management  &  the  Drive  Towards  Commonality  in  a  SoS  World 

One  of  the  more  challenging  of  the  SoS  systems  engineering  tasks  is  the 
management  of  requirements.  This  is  particularly  acute  in  acknowledged  SoS,  where  the 
PM  does  not  have  complete  control  of  many  of  the  constituent  systems  that  are  integrated 
into  the  SoS.  For  the  program,  this  issue  was  made  more  acute  by  the  initialization  of  the 
program  on  a  “come  as  you  are  basis”  based  on  the  desire  to  explore  the  concept  of 
modular  MPs  supporting  a  Seaframe  using  defined  interfaces  before  fully  investing  in  a  full 
up  acquisition  program.  This  resulted  in  technologies  being  selected  to  satisfy  the  original 
performance  requirements  on  a  package-by-package  basis.  As  the  entire  LCS  program 
(Seaframes  and  MMs)  developed,  this  concept  of  an  experimental  development  morphed 
into  a  traditional  acquisition  program.  The  legacy  of  the  initiation  was  a  set  of  MPs  with 
minimal  commonality  between  the  packages,  as  indicated  on  the  left  side  of  Figure  5.  This 
approach,  if  continued,  would  result  in  increased  LCCs  for  the  support  of  the  three  MPs. 
While  the  way  this  issue  was  initiated  for  the  LCS  is  slightly  unique,  the  challenge  of  seeking 
to  reduce  LCC  for  a  SoS  in  which  the  PM  may  not  have  direct  control  over  the  individual 
MSs  and  their  designs  is  not. 


The  program  is  addressing  this  problem  through  the  implementation  of  a  structured 
and  controlled  process  designed  to  introduce  increasing  levels  of  commonality  across  the 
various  mission  packages.  The  objective  is  to  achieve  a  more  flexible  and  controlled 
requirements  and  specification  environment  throughout  the  lifecycle  development  of  MPs. 
To  do  this,  the  PMS  420  Systems  Engineering  Integrated  Product  Team  (SE-IPT) 
established  the  objective  of  moving  towards  a  target  documentation  structure,  illustrated  in 
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the  right-hand  side  of  Figure  5.  This  structure  is  designed  to  provide  a  migration  path  over 
time  towards  common  capabilities.  Initially,  each  package  defined  its  separate  capabilities 
with  a  package-level  performance  specification.  Due  to  the  initial  compressed 
developmental  schedule  and  the  drive  to  use  existing  Program  of  Record  solutions, 
duplicative  or  near  duplicative  common  capabilities  were  often  observed  across  the  other 
MPs,  as  illustrated  on  the  left-hand  side  of  the  figure. 


Figure  5.  Modularizing  the  Structure  of  the  Documentation  Database 

PMS  420,  through  the  SE-IPT,  has  established  a  process  to  define  commonality  and 
modularize  capabilities  to  support  consistent  re-use  across  MPs.  In  the  first  iteration  of  this 
process,  several  common  MP  components  were  isolated  and  established  as  a  common  set 
of  requirements  at  the  next  level  of  specification  below  the  governing  requirements 
documentation.  This  adds  a  common  layer  that  spans  capabilities  across  individual  MS 
boundaries,  as  illustrated  on  the  right  side  of  Figure  5,  modularizing  the  designs  and 
fostering  re-use  of  capability.  This  objective  was  implemented  early  in  package 
development,  even  while  the  three  MPs  were  being  developed  under  an  accelerated 
schedule.  While  the  limitations  in  technology  and  time  prevented  the  implementation  of 
desired  common  capabilities  such  as  a  common  Unmanned  Surface  Vehicle  (USV)  between 
the  MCM  and  ASW  MPs,  some  commonality  was  achieved  and  implemented,  including  the 
design  and  use  of  a  common  USV  cradle,  common  aviation  support  container  designs,  and 
other  components. 

In  other  areas,  while  the  need  for  commonality  across  all  three  MPs  was  defined 
early,  the  ability  to  reach  a  common  capability  took  time.  These  common  capabilities 
included  a  modularized  Mission  Package  Computing  Environment  (MPCE),  and  a  common 
off-board  unmanned  vehicle  communications  capability,  the  Multi-Vehicle  Control  System 
(MVCS).  Each  of  these  common  capabilities,  i.e.,  MPCE  has  its  own  System/Subsystem 
Specification  (SSS)  and  lower-level  documentation.  While  PMS  420  designated  these  as 
common  cross  package  capabilities  to  fulfill  the  identified  needs  of  the  three  MPs,  they  have 
matured  into  requirements  constraints  on  the  existing  packages.  PMS  420  uses  the 
development  of  a  common  set  of  interface  requirements  for  defining  the  trade  space  for 
evaluating  the  value  of  incorporating  new  technologies  within  existing  MPs. 
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This  methodology,  used  for  developing  and  transitioning  MPCE  and  MVCS  is  now, 
through  the  use  of  an  established  requirements  management  and  allocation  process,  being 
extended  to  extracted  common  requirements  and  supporting  capabilities  in  such  areas  as 
Integrated  Logistics  Support  (ILS),  Safety,  Shock  and  Vibration,  and  others.  As  this  method 
continues  to  mature,  it  will  provide  the  basis  for  defining  the  totality  of  the  interface 
requirements  that  a  new  mission  package  would  need  to  met  to  be  effectively  and  efficiently 
integrated  into  the  LCS  MP  SoSs.  The  next  phase  beyond  that  will  be  to  extend  the  push  for 
commonality  down  to  the  MM  and  MS  levels  to  include  the  identification,  development,  and 
specification  of  common  MSs  across  the  MPs,  including  shared  capabilities  such  as  remote 
unmanned  vehicles.  It  is  anticipated  that  this  will  allow  for  increased  modularity  and  re-use 
of  other  Commercial  Off  The  Shelf/Government  Off  The  Shelf  (COTS/GOTS)  technology. 

Steady  progress  has  been  made  in  reaching  this  planned  requirements  structure  of 
the  DOORS  database.  The  requirements  allocation  analyses  for  the  Flight  0  ODD,  Flight  0+ 
ODD  and  the  LCS  Interface  Control  Document  (ICD)  have  been  completed  and  the  Level  2 
MPCE  and  MVCS  SSS  documents  and  the  Level  3  MP  SSS  documents  are  in  the  final 
stages  of  PMS  420  Configuration  Control  Board  (CCB)  approval.  Formal  configuration 
management  (CM)  process  are  used  in  updating  and  tracing  the  SSS  documents  to  the 
CDD  and  ICD  allocated  requirements,  which  provides  impact  and  traceability  analysis 
capability  to  the  MP  level. 

Cost  Prediction  and  Monitoring  of  SoS 

PMS  420  has  been  developing  a  SoS  Online  PM  Tool  that  significantly  improves  the 
ability  to  manage  the  program.  Its  advantages  are  numerous,  including: 

a.  Earned  Value  Management  at  all  SoS  levels,  including  the  ability  to  examine 
the  data  across  various  cross-sections  of  the  program,  and  forecast  cost  and 
schedule  overruns  at  an  early  stage; 

b.  Support  of  multi-year  planning  of  research,  development,  procurement,  and 
sustainment  efforts; 

c.  Risk  management  that  associates  risk  with  cost  and  impact; 

d.  Integrates  a  formal  program  change  management  process  across  all  levels  of 
the  program; 

e.  Allows  senior  program  management  the  ability  to  conduct  what-if  financial 
impact  analysis  without  changing  the  baseline  [what's  the  impact  if  system 
(A)  is  replaced  with  system  (B)]; 

f.  Links  the  Integrated  Master  Schedule  (IMS)  to  task  planning  and  task 
execution  both  at  the  system  level  and  the  SoS  level; 

g.  Links  the  Acquisition  Program  Baseline  (APB)  to  execution  data.  This  gives 
the  program  manager  critical  insight  into  how  lower-level  system  performance 
will  impact  the  overall  success  of  the  SoS;  and 

h.  Integration  of  the  System  Maturity  Model  into  the  Earned  Value  Management 
System  (EVMS). 

PMS  420  uses  this  tool  in  its  monthly  drumbeat  process,  shown  in  Figure  6.  The 
web-accessible,  CAC-enabled  Online  PM  Tool  integrates  program  planning  and  execution 
data  (cost,  schedule,  and  performance),  ensuring  proper  visibility  into  who  is  doing  what, 
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how  well  it  is  being  done  and  what  has  to  be  done  next.  Detailed  task  planning  and 
execution  data,  coupled  with  clever  organizational  and  reporting  capabilities,  have  allowed 
the  PM  to  connect  program  priorities  and  goals  with  past  EV  performance  and  future  EV 
forecasting.  The  Online  PM  Tool  provides  the  framework  and  business  rules  to  establish 
and  maintain  process  discipline.  PMS  420,  executing  activities,  and  cross-functional 
stakeholders  all  have  a  “seat  at  the  table.”  The  real-time  data  availability  promotes  two-way 
communication  and  concurrence,  enabling  a  successful  egalitarian  approach  that  would  not 
be  possible  without  the  open  web-based  tool  environment.  This  provides  the  PM  with 
insight  into  potential  challenges  and  has  helped  avoid  and/or  minimize  program  errors. 


Figure  6.  Monthly  Drumbeat  Process,  Work  Performance  Insight  and  Reporting 

The  Online  PM  Tool  has  been  developed  using  a  process  that  incorporates  key 
stakeholder  requirements.  Processes  are  mapped  to  distinct  decision  points  and  controls 
implemented  to  ensure  results  are  achievable.  The  development  team  used  an  incremental 
build  methodology  to  develop  the  Work  Breakdown  Structure  (WBS),  record  processes, 
review  tool  development  and  obtain  user  feedback.  The  PMS  420  WBS  is  used  as  the 
unifying  element  for  program  information  and  all  information  in  the  Online  PM  Tool  is 
associated  with  a  specific  WBS  element.  Individual  task  statements  are  assigned  a  specific 
WBS  element  and  funding  is  then  allocated  to  executing  activities  associated  with  the 
specific  task  statements.  Execution  data  (earned  value  data  and  variance  analyses)  is 
reported  against  specific  WBS  elements,  which  are  then  displayed  in  EV  reports.  The 
detailed  use  of  EV  and  detailed  work  plans  identifies  areas  that  require  closer  scrutiny,  and 
have  provided  the  PM  with  increased  insight  into  work  performance,  as  shown  in  Figure  7. 
Areas  with  excess  or  un-executable  funds  are  also  more  easily  identifiable,  allowing  those 
funds  to  be  reallocated  to  other  program  priorities,  as  required. 
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Figure  7.  Work  Performance  Insight 

PMS  420  uses  the  Online  PM  Tool  to  display,  discuss  and  digest  data  during 
monthly  drumbeat  meetings.  This  has  enabled  the  monthly  drumbeat  to  evolve  from  a  tool¬ 
centric  execution  status  meeting  to  a  more  generalized  program  management  meeting 
where  Cost,  Schedule,  Risk  and  Performance  are  reviewed.  This  meeting  has  come  to 
replace  other  routine  meetings,  concentrating  PM  efforts  into  a  single  day  and  freeing 
program  office  staff  to  do  other  work.  Snapshot  documents,  i.e.,  program  Cost  Performance 
Report  and  Integrated  Master  Schedule,  are  created  using  the  tool  to  summarize  and 
synthesize  data  into  information,  record  this  information  for  historical  purposes,  and  provide 
senior  management  with  real-time  status  of  program  health. 

PEO-level  reporting  is  accomplished  through  a  PEO  Dashboard,  which  provides 
senior  executive-level  insight  into  all  programs  in  the  PEO.  Senior  management  has  the 
ability  to  view  planning  and  execution  performance  data  across  program  offices,  executing 
and  performing  activities,  contractors,  enterprise  mission  capabilities,  lifecycle  stages,  and 
across  SoS  and  Families  of  Systems.  PEO  Ouarterly  Execution  Reviews  (OER),  previously 
an  arduous  process  of  data  collection  and  validation,  are  accomplished  through  the  use  of 
the  PEO  Dashboard  and  Online  PM  Tool.  The  PM  can  generate  a  PEO-level  summary  of 
the  same  data  used  to  manage  the  program,  and  the  PEO  can  request  deep-dives  into 
areas  of  interest  or  concern. 


Understanding  and  Influencing  SoS  Reliability 


Availability  is  a  more  complex  problem  for  the  SoS  than  in  traditional  systems.  This 
arises  primarily  out  of  two  attributes  of  many  SoSs;  the  use  of  modular  systems  design,  and 
the  ability  of  the  SoS  to  accomplish  multiple  mission  capabilities  using  various  components 
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of  the  SoS.  The  program  has  been  addressing  this  availability  issue  through  several 
analysis  approaches  and  the  development  of  tailored  methods  linked  to  the  planned 
operational  concepts  of  the  MP  SoSs. 

The  use  of  traditional  availability  and  reliability  tools,  such  as  reliability  block 
diagrams  (RBDs),  Failure  Mode  Evaluation  and  Criticality  Analysis  (FMECA),  and  others  are 
well  understood  and  there  is  much  experience  in  industry  with  their  application.  However,  in 
a  SoS,  these  methods  are  faced  with  two  added  demands.  First,  the  introduction  of 
modularity,  open  system  design,  and  remote  operations  produce  increasing  number  of 
components  and,  therefore,  more  opportunity  for  component  to  component  failures.  Some  of 
the  impacts  on  operational  availability  in  such  extended  systems  are  indicated  in  the  list 
below: 

•  The  mission  component  string  is  inherently  less  reliable  because  we  increase 
the  number  of  serial  components  in  the  mission/operational  function; 

•  Extended  Unmanned  systems  require  set  up  time  and  the  potential  for 
damage  is  increased  because  of  the  increased  handling,  in  addition  the 
deployment  and  recovery  environment  and  handling  systems  design 
introduce  opportunities  for  damage; 

•  Infrastructure  Over-head  can  be  over  whelming  in  the  particular  adaptation  of 
modular  Plug  and  Play  (P&P)  design  approach  (weight,  extra  services, 
handling  operations,  software  and  hardware  overhead);  and 

•  Deployment  of  remote  systems  have  security  challenges  (physical  and  data 
related). 

The  second  added  demand  is  that  of  multiple  mission  capabilities  and  flexibility  in 
configurations  to  achieve  them.  Simply  stated,  we  typically  have  more  capabilities  provided 
by  the  SoS,  and  can  execute  the  capability  with  several  combinations  of  components.  Some 
of  the  challenges  introduced  by  this  flexibility  are  described  in  the  list  below: 

•  Operational  use  is  constrained  during  a  specified  time  period; 

•  Operational  use  may  use  a  small  percentage  of  the  mission  suite,  depending 
on  the  mission,  e.g.,  for  MCM,  mapping,  identification,  clearing;  and 

•  Operational  environment  may  call  for  a  different  subset  of  equipment  to  be 
used  in  a  deployment. 

In  the  case  of  the  program,  we  have  the  following  specific  issues  that  we  must 
address: 

The  majority  of  the  elements  of  a  MP  are  different  and  widely  distributed 
interfaces  will  affect  the  availability; 

The  organic  off  board  vehicles  can  be  utilized  differently  each  mission; 

The  utilization  of  mission  equipment  per  mission  will  vary — alternative 
mission  equipment  may  be  substituted; 

The  deployment  and  utility  times  will  vary  based  on  operational  mission 
goals; 

Ship  Availability  is  a  component  of  the  MP  architecture;  and 
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•  Software  reliability  in  MPCE  and  MVCS  needs  to  be  considered. 

An  additional  issue  for  the  program  (and  becoming  prevalent  in  other  SoS)  is  that  the 
requirement  for  MP  availability  is  defined  in  terms  of  a  Materiel  Availability  KPP.  Materiel 
Availability  (Am)  for  the  LCS  MPs  is  established  with  a  threshold  of  0.64  and  an  objective  of 
0.712.  There  is  no  specified  separate  requirement  for  the  traditional  A0,  since  the  LCS  CDD 
indicates  that  “it  is  embedded  in  the  Materiel  Availability  KPP.”  A  novel  methodology  has 
been  applied  to  analyze  and  decompose  Am,  resulting  in  an  approach  that  separates  Am  into 
two  components.  The  first,  called  Active  Availability  (Aa),  is  a  factor  that  is  a  function  of  fleet- 
level  support  design,  including  such  components  as  depot-level  repair  requirements, 
fielding,  deployment,  and  support  strategies.  The  second  is  the  traditional  A0,  defined  at  the 
fleet  level  and  computed  as  the  average  of  squadron  or  unit  level  A0. 

In  this  fashion,  a  clear  definition  of  Am  for  the  program  is  developed  and  represented 
by  Am  =  Aa  *  A0.  Monte  Carlo  simulations  and  support  parameter  investigations  are  being 
used  to  determine  the  impact  on  MP  Am.  Initial  results  indicate  that  target  parameters  for  Aa 
and  A0  with  their  impact  on  Am  could  be  established  as  given  in  the  example  in  Table  1 . 


Table  1.  Example  LCS  MP  Am  Composition 


CDD  Requirement 

Target  Aa 

Target  A0 

Target  Am 

Threshold  .64 

0.887 

0.738 

0.655 

Objective  .712 

0.887 

0.809 

0.718 

Using  this  Am  decomposition  and  simulation  method,  it  is  possible  to  establish 
requirements  for  both  Aa  and  A0  that  can  be  allocated  to  the  MPs  and  further  decomposed  to 
their  individual  MM  and  MS  components.  The  primary  use  of  the  Am  decomposition  method 
is  to  establish  support  concepts  and  strategies  that  will  establish  a  defined  level  of  Aa.  After 
Aa  is  clearly  established,  A0  can  be  determined  using  the  equation  A0  =  Am  /  Aa.  This 
provides  target  A0  requirements  that  can  be  allocated  to  the  MPs  and  their  components.  To 
address  the  above  issues,  Operational  component  strings  for  the  various  mission  functions 
are  defined.  An  example  of  this  is  shown  in  Figure  8 
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Figure  8.  Mission  Function  Component  Strings 


The  following  steps  are  then  performed  on  the  individual  mission  strings: 

•  Operational  strings  were  analyzed  to  identify  the  components  required  to 
execute  independent  mission  functions  of  the  system; 

•  An  assessment  of  the  string  to  achieve  a  Mission  A0  contribution  is 
performed; 

•  Common  components  (nodes)  that  form  a  critical  function  in  more  than  one 
mission  function  are  identified,  and  operational  time  is  calculated  for  each 
mission  it  touches  over  the  deployment  cycle;  and 

•  Allocation  of  the  Mission  A0  decomposes  to  an  A0  requirement  at  the 
component,  Line  Replaceable  Unit  (LRU),  level. 

Currently,  the  program  is  evaluating  the  use  of  the  mission  function  strings, 
decomposition  of  the  A0  for  the  mission  to  individual  components,  and  a  Monte  Carlo 
simulation  for  mission  A0  analysis.  The  goal  is  to  determine  individual  component  A0 
requirement  targets  and  determine  the  impact  of  these  on  MP  A0  and,  therefore,  on  the  Am 
threshold  and  objective  requirements. 

Expanding  the  Tool  Set — Technology  Trades  &  SoS 
Performance  Predictions 

The  existing  tool  set  has  been  invaluable  to  enabling  the  PM  to  gain  increased 
insights  into  the  developmental  status  and  risks  associated  within  a  complex  SoS.  PMS  420 
has  now  begun  working  to  expand  upon  the  foundation  of  those  tools  and  the  system 
readiness  monitoring  capabilities  provided  by  implementation  of  the  SMM  methodology  and 
is  seeking  to  expand  the  technical  and  programmatic  tools  and  methodologies  available  for 
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the  PM.  Specifically,  there  exists  the  need  for  tools  to  assist  the  PM  in  understanding  the 
impacts  of  technology  insertion  options  and  to  gain  insight  into  predicted  SoS  performance, 
enabling  the  more  effective  conduct  of  tradeoff  analysis.  This  analysis  would  assess  SoS 
performance  and  capability  objectives  and  provide  recommendations  enabling  the  PM  to 
make  choices  that  optimize  the  SoS  on  the  basis  of  cost,  technical  risk,  or  anticipated 
performance. 

Technology  Insertion  &  SoS  Analysis 

One  of  the  primary  benefits  of  the  modular  approach  to  SoS  development  is  that  new 
technologies  capabilities  can  be  rapidly  incorporated  to  improve  reliability,  performance,  or 
reduce  LCCs.  However,  comparable  technologies  when  integrated  may  result  in  significantly 
different  integration  risks,  performance  impacts  across  the  SoS,  and  reliability  or  cost 
impacts.  How  to  decide  on  what  technologies  to  change  and  which  technologies  to  select  is 
an  area  of  critical  interest  to  PMS  420  as  the  MPs  mature  from  development  and  enter  their 
operational  lifecycles. 

As  the  SoS  technologies  reach  their  designed  or  actual  limits  (of  cost,  performance, 
etc.),  they  should  be  reviewed  for  replacement  by  newer  or  more  robust  technologies  that 
would  provide  improvement  to  the  SoS  optimization,  performance,  and  capabilities.  The 
existing  methodologies  and  incorporation  of  tools  developed  to  date  by  PMS  420  have 
provided  the  program  with  an  unprecedented  view  into  the  details  of  the  status  of  the  SoSs. 
The  difficulty  lies  with  combining  the  various  details  to  provide  the  PM  with  a  current 
composite  view  into  the  SoS  to  support  analysis  and  impacts  related  to  changing 
technologies  within  the  existing  SoS.  A  composite  view  would  enable  the  PM  to  determine  if 
a  proposed  Technology  would  generate  performance  beneficial  or  detrimental  to  the  SoS, 
would  have  a  budget  that  is  proportional  to  established  values — that  is,  reaching  the  end  of 
its  lifecycle —  would  bring  too  much  risk  to  the  SoS,  would  be  within  the  physical  constraints 
(size,  mass,  etc.)  of  the  SoS,  and  more. 

The  question  is  how  to  review  several  technologies  that  could  be  added  to  the  SoS 
and  determine  the  best  candidate  based  upon  the  desires  of  the  PM.  To  make  this  decision 
consistently  and  without  bias,  a  modified  version  of  the  Analytical  Hierarchical  Process 
(AHP)  will  be  utilized.  With  AHP,  PMS  420  can  assign  several  categories  and  sub¬ 
categories  that  the  various  technologies  will  be  rated  against,  such  as  Cost,  Physical 
Constraints,  SRL,  Reliability,  etc.  The  scores  from  the  categorical  comparison  of  the 
technology  ratings  will  then  be  weighted  by  the  needs  and  desires  of  the  PM  (e.g., 
budgetary  constraints).  The  result  will  be  a  list  of  the  current  and  potential  technologies 
ranked  in  order  of  recommended  choice.  A  basic  example  of  this  hierarchical  calculation  is 
shown  in  Figure  9. 

Technology  Analysis  and  Insertion  Tool  Development 

The  difficulty  with  utilizing  the  technology  analysis  and  insertion  calculation  is  its 
complexity  and  the  need  to  allow  for  easy  modification  to  the  weighting  parameters  and 
ratings  of  technologies.  In  the  complex  world  of  program  acquisition,  there  are  times  that 
this  calculation  will  need  to  be  finished  in  a  relatively  short  period  of  time,  demanding  an 
implementation  that  is  prompt,  dependable,  unbiased,  and  accurate.  Much  of  the  data  that 
is  needed  for  the  calculation  is  spread  out  in  many  tools  and  locations  and  would  need  to  be 
accessed  for  the  calculation  to  produce  meaningful  results. 
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Figure  9.  Technology  Analysis  Hierarchy 

The  solution  to  these  issues  is  to  maximize  the  usage  of  the  current  tools  while 
minimizing  the  footprint  of  a  new  one.  We  have  been  working  with  the  SIT  to  develop  and 
test  a  SRL  calculation  plug-in  for  an  architecture  tool,  as  a  distributable  example.  Building 
off  of  this  starting  point,  we  have  designed  this  trade-off  tool  to  act  as  a  substantial  plug-in  to 
the  program’s  already  established  architecture  modeling  tool.  By  leveraging  the  SoS 
architecture  that  is  already  developed  and  being  maintained  and  by  adding  small  changes  to 
the  architecture  tool’s  data  model,  the  new  trade-off  tool  will  function  with  the  established 
architecture  tool,  greatly  reducing  the  learning  curve. 

The  small  modifications  to  the  architecture  tool  data  model  will  not  impact  any  of  the 
existing  programmatic  data  that  is  currently  captured,  nor  will  it  impact  the  normal 
architecture  review  cycles.  (As  the  data  model  can  easily  be  expanded  to  incorporate  the 
new  fields,  it  can  be  designed  to  fit  into  DODAF  1  .x  and  2.0  versions.)  The  modified  fields 
will  capture  the  ranking  data  that  relates  to  the  Technology  Analysis  and  Insertion 
calculation  in  the  configuration  controlled  environment  of  the  Architecture  Model.  The 
information  for  technologies  that  are  not  currently  part  of  the  SoS  would  be  populated  into 
the  architecture  tool.  A  small  modification  to  SIT’s  existing  SRL  calculation  plug-in,  will 
enable  it  to  work  in  this  design  (different  architecture  tool’s  commands).  The  weighting 
values  for  the  criteria  will  be  entered  by  the  PM  and  securely  stored.  The  outcome  of  the 
calculation  will  be  a  ranked  list  of  choices  for  the  position  within  the  SoS.  A  basic 
representation  of  the  tool  is  shown  in  Figure  10. 
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Figure  10.  Technology  Analysis  and  Insertion  Tool 

Prediction  of  Performance  Using  a  Performance  Level  Monitoring 
Methodology 

Through  system  development,  PMs  are  expected  to  quantifiably  justify  that  their 
program  will  result  in  the  delivery  of  a  system  with  the  required  performance.  The  traditional 
PM  has  several  technical  and  program  management  tools  at  their  disposal,  including  TPMs, 
Modeling  and  Simulation,  etc.,  that  provide  insight  and  predictive  capability  in  system 
performance.  When  the  program  matures  to  a  point  at  which  actual  test  data  can  be 
gathered,  it  is  compared  against  expected  system  performance.  Due  to  the  complex  nature 
of  SoS  interdependencies,  PMs  are  especially  challenged  when  asked  to  quantifiably  justify 
the  investment  in  time,  personnel,  financial,  and  material  resources  in  the  program  during 
SoS  development. 

Traditional  program  and  technical  management  tools  must  be  extended  to  provide 
the  necessary  insight  to  the  acknowledged  SoS  environment.  Given  that  the  performance  of 
a  SoS  is  directly  dependent  upon  the  performance  levels  of  the  individual  systems 
composing  the  SoS,  as  these  capabilities  are  being  independently  developed  by  PARMS 
(over  which  he  has  limited  directional  authority  and  who  may  be  developing  the  capabilities 
to  fulfill  a  different  set  of  performance  metrics  or  may  be  unwilling  to  share  detailed  technical 
status  with  external  organizations).  Even  where  the  individual  MS  performance  may  directly 
translate  to  a  SoS  KPP  area,  the  nature  of  MP  SoS  and  its  Concept  of  Operations  does  not 
mean  that  it  provides  the  total  answer.  In  various  scenarios,  the  individual  MP  KPPs  can  be 
achieved  through  using  various  combinations  of  the  systems  within  the  SoS.  For  example, 
the  LCS  may  decide  to  engage  Fast  Inshore  Attack  Craft  (FIAC)  with  either  the  LCS’s  core 
gun,  capabilities  onboard  the  MH-60  helicopter,  the  30mm  Gun  Mission  Module  (GMM)  (of 
the  SUW  MP),  or  eventually  the  missiles  of  the  Surface  to  Surface  Missile  MM  (of  the  SUW 
MP).  All  of  which  can  in  full  or  in  combination  satisfy  individual  KPPs  for  the  SUW  MP.  This 
estimation  of  performance  difficulty  is  further  complicated  when  the  SoS  is  being  developed 
in  an  incremental  manner.  Again  using  the  SUW  MP  as  an  example,  the  first  SUW  MP  is 
currently  installed  onboard  LCS-1  and  includes  two  30-mm  Gun  Mission  Modules,  an 
Aviation  (MH-60R  armed  helicopter)  MM,  and  a  prototype  MSM.  The  first  SUW  MP  does  not 
include  the  Surface-to-Surface  Missile  MM  (based  on  the  NLOS-LS),  which  will  be  added  in 
Increment  2.  Understanding  the  capability  provided  by  MP  increments  and  ultimately 
whether  the  baseline  (full  capability)  MP  will  satisfy  the  full  set  of  performance  requirements 
is  of  interest  to  the  PM.  A  final  complication  is  that  when  conducting  SoS  and  incremental 
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acquisition  of  this  nature,  complete  E2E  test  and  evaluation  (T&E)  may  not  be  feasible  and 
computer  intensive  modeling  and  simulation  may  not  be  practical  in  a  schedule  driven 
environment  or  where  the  SoS  PM  may  not  have  the  full  technical  models  of  the  individual 
systems.  So  within  these  limitations,  how  can  the  PM  gain  insight  into,  and  predictive 
capability  for,  determining  the  ability  of  the  SoS  to  achieve  required  performance? 

To  answer  these  questions,  PMS  420,  in  conjunction  with  SSC-Pacific,  Northrop 
Grumman,  and  the  SIT,  has  been  expanding  the  SMM  to  incorporate  a  Performance  Level 
Monitoring  (PLM)  methodology.  The  PLM  is  being  developed  to  understand  if  the 
performance  will  satisfy  the  KPPs  and  to  understand  the  deltas  in  performance  between  the 
initial  MPs  and  later  MP  increments,  which  will  provide  the  full-up  IMP  capability.  Ultimately, 
this  tool  will  also  support  the  analysis  of  mission  threads  using  different  MP  configurations, 
i.e.,  providing  insight  in  performance  capability  of  the  MCM  MP  if  one  of  its  USVs  is  down  for 
maintenance  and/or  as  a  tool  for  evaluating  the  impact  of  incorporating  new  capabilities  or 
changing  existing  capabilities  within  a  MP. 

Performance  Level  Monitoring  Explained 

The  PLM  strives  to  apply  a  modified  TPM  type  approach  to  a  SoS  construct. 
However,  instead  of  focusing  on  a  measurable  technical  value  that  can  be  monitored  during 
development  within  a  individual  system,  the  PML  links  the  SoS  KPPs  to  individual 
component  capabilities,  their  maturity,  and  their  potential  usage.  The  SMM,  Concept  of 
Operations  (CONOPS),  and  usage  rate  variance  analyses  are  all  considered  in  the  PLM 
calculation. 

To  implement  this  process,  significant  up-front  evaluation  will  be  required  by  the  SoS 
program  office.  The  first  step  of  the  methodology  is  to  define  the  SoS  MP  in  terms  of  it 
component  MSs  and  to  map  those  systems  and  their  capabilities  to  their  projected  impact 
on  satisfying  the  MP  KPPs.  The  individual  MS  capabilities  are  then  adjudicated  by  the  SoS 
PM  in  terms  of  their  maturity  and  inclusion  within  an  individual  MP,  breaking  them  into  three 
generic  categories  of  Advanced  Developmental  Models  (ADM),  Engineering  Development 
Modules  (EDM),  and  Production  Models  (PROD)  that  are  mapped  to  their  expected 
maturation  points  over  the  analysis  timeline.  This  adjudication  of  systems  is  represented 
through  the  use  of  a  weighting  function  to  represent  the  individual  capability’s  maturity  (real 
or  anticipated)  for  each  level  of  development.  While  this  method  works  well  for  MS  that  are 
not  integrated  with  others  to  deliver  required  capability,  such  as  the  GMM,  this  becomes 
more  complex  when  two  or  more  systems  must  come  together  to  provide  a  level  of 
capability  such  as  a  MM,  for  example  the  combination  of  the  ASW  USV  MS  and  the  USV 
Towed  Array  System  (UTAS)  to  provide  a  passive  search  capability.  Fortunately,  the 
ongoing  development  of  the  SMM  concept  allows  for  a  potential  approach  by  using  the 
value  calculated  for  the  MM  SRL.  The  individual  technologies  can  then  be  weighted  in  terms 
of  their  contribution  to  the  accomplishment  of  the  capability  and  be  combined  into  a  series  of 
capabilities  or  MM  values.  The  integrated  MM  capabilities  can  be  expressed  as  a  single 
value  where  the  level  of  capability  that  the  module  comprised  of  capabilities  (x,  y,  and  ..)  that 
can  contribute  towards  the  satisfaction  of  the  MP  KPP  requirement  given  the  level  of 
maturity  of  the  capability  in  the  MP. 

The  next  stage  of  the  PLM  is  to  define  the  impact  of  various  CONOPS  on  the  ability 
of  the  SoS  to  satisfy  the  KPP  requirements.  As  one  of  the  strengths  of  a  SoS  is  its  inherent 
flexibility  where  component  systems  can  be  organized  to  solve  the  capability  problem,  this 
can  often  be  translated  to  where  the  individual  capabilities  may  be  used  in  varying  ways  to 
accomplish  the  same  mission.  These  varying  CONOPS  result  in  increased  complexity  for 
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the  analyst  in  trying  to  predict  what  level  of  performance  is  being  achieved.  While  modeling 
and  simulation  tools  can  be  used  to  conduct  this  type  of  analysis,  the  variations  would  make 
this  an  expensive  and  time-consuming  process  for  the  program  and  would  not  provide  the 
PM  with  the  rapid  insight  into  options  that  maybe  required.  To  address  this  issue,  the  PLM 
seeks  to  develop  a  set  of  scenarios  for  each  of  the  MP  SoS  that  represent  the  range  of 
potential  operational  usage  concepts  for  the  individual  capabilities  (or  modules)  within  the 
SoS  as  applicable  to  each  KPP  and  incremental  MP.  This  enables  the  derivation  of  a  set  of 
equations  relating  the  KPPs  to  their  component  technologies  and  to  a  specific  CONOP.  The 
set  of  CONOPS,  each  reflecting  an  anticipated  level  of  performance  and  technical 
maturity/integration  of  a  specific  capability  (x)  at  a  specific  point  (for  this  example  as 
represented  by  a  specific  mission  package)  in  time  (n)  are  then  matrixed  together  to  enable 
a  calculation  of  the  overall  predicated  performance  of  the  SoS  across  a  range  of  scenarios. 
When  conducting  the  analysis,  the  exact  usage  rate  for  each  of  the  MS/MM  may  be 
unknown.  In  this  case,  running  the  analysis  for  each  CONOPS  using  a  minimum,  maximum, 
and  average  anticipated  usage  rate  for  each  capability/module  can  be  used  to  develop  a  set 
of  error  bars  in  performance  predictions.  As  a  tool  for  the  management  of  risk  and  for 
predicting  when  performance  will  be  achieved  or  to  understand  the  potential  impact  of 
changes,  a  graphic  similar  to  the  traditional  TPM  graphic  can  then  be  constructed  by 
calculating  composite  KPP  values  for  each  MP  increment  and  plotting  the  composite  level  of 
performance  against  time. 


1.  Map  Technologies  to  KPPs  and  Maturity 


Goal:  Predict  the  ability  of  a  complex  systems  to  achieve  required  performance 


1.  Map  the  Systems  to  their  impacts 
on  key  performance  parameters 


2.  Map  the  maturity  development  of  the 
Systems  to  the  SoS  development  schedule 


3.  Develop  a  relationship  between  system  usage  satisfying  a  KPP  in  a  SoS  and 
its  maturity  (in  terms  of  a  weighted  value)  against  anticipated  performance 


•  Low  cost,  computation  simple  methodology 
for  rapidly  gaining  insight  into  SoS 
Performance 

•  PLM  may  provide  insight  into  incremental 
capability  compared  with  performance 
requirements. 

•  Evaluation  tool  for  new  capabilities  prior  to 
their  being  incorporated 


4.  Adjust  for  usage  impact  under 
various  employment  options 

CONOPSa,  -  0Tj„  +  yTj. 
CONOPSb.  -  £T|„  +  eTj.  »  yT5„ 
CONOPS;  .  -  dT„  -K  nT;. 


5.  Average  the  results  from 

individual  employment  options  to 
obtain  insight  into  ability  to 
achieve  obtainment  of  the 
desired  performance  parameter 


.  Use  predictions  of  improved 
maturity  (SRL)  over  time  to 
derive  a  predicted  growth  path  of 
performance  for  SoS 


«■ 

• 

-a  -  [CONOPa.  CONOPr.  CONOPrl-AYG<CONOPA-CONOPR-CONOPr> 


3.  Conduct  Variance  Analysis 


Figure  11.  Performance  Level  Assessment  Methodology 

The  intent  behind  the  PLM  is  to  provide  the  PM  necessary  insight  into  incremental 
capability  compared  with  CDD  performance  requirements.  As  with  all  predictive  models,  the 
analysis  will  need  to  be  compared  against  measured  test  data  as  it  becomes  available  to 
verify  predictions  and  identify  if  the  program  is  on  course  to  meet  CDD  performance 
requirements.  Figure  1 1  provides  a  graphical  flow  of  the  methodology  described  above  and 
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a  more  detailed  description  of  this  methodology  can  be  found  in  Notional  Assessment 
Methodology  for  KPP  Accomplishment  in  a  SoS:  Proposed  Methodology  for  Measuring 
Performance  Progress  within  a  System  of  Systems  (SoS)  (Volkert,  2009).  This  methodology 
has  been  further  expanded  upon  by  SIT  at  this  symposium  (Tan,  Sauser,  Ramirez-Marquez, 
Magnaye,  Nowicki  &  Deshmukh,  2010).  A  projected  further  expansion  of  this  methodology 
will  be  to  allow  for  the  evaluation  of  new  capabilities  prior  to  their  being  incorporated  into  a 
planned  upgrade  or  replacement  of  an  existing  capability. 

Conclusion 

The  increasing  use  of  the  System  of  Systems  (SoS)  model  for  the  fielding  of  new  and 
improved  warfighting  capabilities  poses  new  management  challenges  for  the  DoD.  To 
support  the  Program  Manager,  the  US  Navy  (PMS  420,  SSC  Pacific),  Northrop  Grumman, 
and  the  Stevens  Institute  of  Technology  have  been  collaborating  on  the  development  and 
verification  of  a  set  of  comprehensive  financial  and  portfolio  management  methodologies  for 
acknowledged  SoSs.  The  tools  and  capabilities  that  are  being  developed,  discussed,  and 
expanded  by  PMS  420  reflect  the  real-world  challenges  facing  a  SoS  PM  and  reflect 
valuable  lessons  learned  to  date  within  the  LCS  Mission  Modules  program.  Starting  with  the 
field  of  technology  maturation,  the  team  has  developed  the  standard  TRL  methodology  into 
a  concept  that  develops  a  System  Readiness  Level  (SRL)  measurement  as  a  measure  of 
SoS  integration  and  maturity.  This  methodology  has  been  demonstrated  and  has  been  used 
as  the  developmental  springboard  into  an  approach  for  determining  and  predicting  the 
probability  of  achieving  system  performance  and  for  understanding  the  impact  of  technical 
option  trades.  Financial  tools  have  been  developed  and  implemented  that  allow  the  PM  to 
gain  insight  beyond  that  afforded  by  traditional  EV  reporting  and  that  can  provide 
management  assistance  in  resource  allocation  in  dynamic  programs.  The  maturation  of  the 
requirements  management  process  for  a  SoS  was  discussed  and  the  capabilities  of  using  it 
as  a  tool  for  reducing  Life  Cycle  Cost  presented.  The  process  described  dictates  the  need 
for  a  methodology  for  SoSs  that  allows  for  fully  integrated  analysis  and  trade-offs  of  the 
technical,  cost,  and  schedule  design  spaces.  While  SoS  show  great  promise  for  providing 
flexible  and  cost-effective  provisioning  of  capabilities  to  the  DoD,  the  evolution  of 
management  tools  will  need  to  continue  to  advance  in  order  to  allow  for  the  more  efficient 
application  of  scarce  resources  from  the  conception  of  program  initiation.  Otherwise,  SoS 
Program  managers  may  be  forced  to  continue  to  face  many  of  the  challenges  PMS  420  has 
been  through  and  will  need  to  expend  resources  in  solving  those  management  challenges 
vice  applying  the  resources  to  product  development. 
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Creation  of  a  System  of  Systems 
Portfolio  Management  and  Technology 

Selection  Methodology 
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Overview 


•  System  of  Systems  Management 

-  Acknowledged  SoS  Example  Mission  Modules  Program 


•  Systems  Engineering  Management  and  Insight  in  SoS  Programs 

-  SRL  Used  to  Determine  SoS  Maturity  Analysis 

-  Requirements  Management  &  the  Drive  Towards  Commonality  in  a  SoS  World 

-  Cost  Prediction  and  Monitoring  of  SoS 

-  Understanding  and  Influencing  SoS  Reliability 


•  Expanding  the  tool  set  -  Technology  Trades  &  SoS  Performance 
Predictions 

-  Technology  Insertion  &  SoS  Analysis 

-  Technology  Analysis  and  Insertion  Tool  Development 

-  Prediction  of  Performance  Using  a  Performance  Level  Monitoring  Methodology 


•  Conclusion 


SoS  Acquisition  Challenges 


•  SoS  acquisition  management  -  a  significant  increase  in 
complexity  over  traditional  system  acquisition 

•  Development  requires  that  significant  numbers  of 
technologies  be  integrated  to  one  another 

•  Challenges  traditional  development  monitoring  tools  and 
cost  models 


-  need  to  capture  integration  complexity 

-  level  of  effort  required  to  connect  individual  components 

•  Unintended  Consequences  -  high  degree  of  inter-linkage 
between  components  can  cause  unintended  impacts  to 
overall  system  performance 

-  components  are  modified  from  original  use 

-  Technology  change:  replaced  throughout  the  system  life 
cycle 


The  result  of  this  acquisition  management  paradigm  shift  has 
been  significant  schedule  and  cost  overruns  in  SoS  programs 
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PMS  420  Tasking-  Providing  Focused 
Warfighting  Capabilities 
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Sensors  Weapons  Vehicles 


What  is  a 


Package? 


COBRA  UDS 
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RMMV 

C. 


{Mission 
Systems 


Mission  Package 


Support  _  Mission 
Equipment  —  Modules 


MPCC 

MVCS 


Crew  & 

Support  Aircraft 


Support  Containers 
Support  Equipment 
Standard  Interfaces 


MPAS 


VTUAV 


_ 

Detachments 
Mission  Modules 
Aviation 

h-60 


Program  Office  Role  and  Needs 


PEO  LMW  /  PMS  420  is  responsible  for  the  development,  acquisition,  and 
sustainment  of  modular,  swappable  Mission  Modules  to  be  used  on  the 
Littoral  Combat  Ship  (LCS) 

Mission  Modules  leverage  considerable  amounts  of  technology  from 
existing  programs  of  record,  while  also  requiring  development  of  new 
integration  software  and  components 


•  Keys  aspects  of  the  project  include  not  only  monitoring  the  status  of 

technology  development,  but  also  the  maturity  of  the  numerous  integrations 
between  those  technologies  and  external  interfaces 
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This  has  resulted  in  a  very  complex  and  diverse  System  of  Systems  (SoS) 
engineering  activity  with  a  need  to  obtain  quick  and  accurate  snapshots  of 
development  maturity  status,  risks,  and  performance 


Coordination  of  Multiple  Independent  Programs  to  meet  PMS  420  Needs 
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SoS  Organizational  Challenges 


PEO  A  (NAVAIR)  is  responsible  for  the  acquisition  of  air 
ASW,  assault  &  special  mission  programs. 

PEO  U&W  (NAVAIR)  is  responsible  for  the  acquisition  of 
unmanned  aviation  and  strike  weapons. 


PMA  299:  MH-60  Helicopters 
PMA  266:  Unmanned  Air  Systems 


PEO  SHIPS/The  LCS  program  office  (PMS  501)  exercises 
technical  and  programmatic  oversight  of  the  industry  teams 
via  a  comprehensive  team  representing  all  systems 
engineering  disciplines. 


PMS  501:  Littoral  Combat  Ships 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  T»C  NAVY 
(RESEARCH,  DEVELOPMENT  AND  ACQUISITION) 
PEO  STRUCTURE 


PMS  420:  LCS  Mission  Modules 


PMS  403:  Unmanned 
Maritime  Vehicles 
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Developmental  Challenges 
Incremental  Development 


COBRA 


Increment  1 

MP  1  in  2007  and 
MP2  in  2009 

Increment  2 

MP3  in  2012  and 
MP4  in  2013 

Increments 

MP  5  In  2014  and 
MP  6  in  2015 

Increment  4 

BASELINE 

MP  7+  in  2016 

TRL 

RMMV  (x2) 

RMMV  (x2) 

RMMV  (x2) 

RMMV  (x2) 
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AN/AQS-20A  (x3) 

AN/AQS-20A  (x3) 

AN/AQS-20A  (x3) 

AN/AQS-20A  (x3) 

8 

AMNS  (EDM  xl) 

AMNS  (xl) 

AMNS  (xl) 

AMNS  (xl) 

7 

ALMDS  (xl) 

ALMDS  (xl) 

ALMDS  (xl) 

ALMDS  (xl) 

7 

USVwUSS(xl) 

USVwith  US3not 
ready  for 

procurement  due  to 
developmental  and 
programmatic  issues 

USVwUSS(xl) 

USVwUSS(xl) 

6 

Surf/  beach  zone 
detection  capability 
introduced  with 
VTUAV/C0BRA 

COBRA  (xl) 

COBRA  (xl) 

COBRA  (xl) 

8 

Influence  mine  sweep  capability  is  increased  with 
the  MH-60/  OASIS 

OASIS  (xl) 

OASIS  (xl) 

6 

OASIS  mounted  on  helo 


RAMICS  mounted  on  helo 
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Near-surface  neutrali 


Future  MCM  MP  Increments  are  subject  to  POR 
schedule  slips  that  aren't  under  PMS  420  direct  control 
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SoS  Challenges  -  Modularity  and 
Commonality  throughout  the  SoS 


PMS  420  LCS  Mission  Modules 


Mission  Modularity  through  SoS 
Implementation 


PMS  420  LCS  Mission  Modules 


I  $  £  Vehicle 
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AUVSI 


RMV  UUV  USV  VTUAV 

MCM  MCM  MCM  mcm 


Implementing  Commonality 
Throughout  the  SoS 


Sensor  I  Weapon 


I 


M»f»R 

VTUAV 

AQS-20A  ■  m 

Mn<  rtmnifl  jomm  Gun 

Mine  IdmMcaMon  '  L 


Improved  Commonality  viewed  as 
key  to  LCC  reductions  and  towards 
Improved  multi-functional  capability 
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System  Maturity  Model  (SMM)  Methodology 


1.  Develop  System  Architectures 


2.  Determine  Criticality 


Physical 

Soft  ware/  Hard  ware 

System  architecture  provides  the  foundation  for 
system  maturity  assessments 


3.  Build  Assessment  Process 

•  Customize  applicable  TRL/  IRL  criteria 

•  Build  SRL  advancement  schedule 

•  Tie  criteria  to  program  test  events  / 
milestones 


Systems  *  Review  proposed  criteria,  schedule,  and 

Engineering  milestones 

IPT 


Systems 

Engineer 


Identification  of  critical  elements 
and  interfaces  to  be  evaluated 


* 

PM 


•  Approve  assessment  framework 


Architectures  and  framework  are  locked  after  approval  and  will  remain  so  unless  the  program  is  re-baselined 


4.  Conduct  System  Maturity  Analysis  w/  SRL 


y— BBSS— 

♦ 

iHHH- 

.»•  "-S- 

•  *-  u. 

tin 

.  *  r.,Tai 

T  r  t 

Evaluate  and  Justify  TRLs  / 
IRLs 


Calculate  SRL 


Build  Maturity 
Reports 


o 


_ Iterate 

Identify  Risks  Against  Schedule 

SRLassessment  and  test  events  /  milestone  gates  are  at  or  in  advance  of  schedule 

SRlassessment  is  at  or  in  advance  of  schedule,  but  test  events  /  milestone 
gates  remain  to  be  closed 

SRLassessment  and  test  events  /  milestone  gates  are  behind  schedule 

Iterate 


5.  Interpret  and  Apply  Results 


ssbJ 
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Maturity  Analysis  Outputs 


•  »•••«*  *•*  »  • 
EVMSand  Schedule  Data 
Inserted 


Outputsof  the  analysis  are  analyzed  against  projected  cost 
and  schedule  data  to  determine  current  development  status 

Future  planning  can  also  be  conducted  through  trade-off 
analyses  and  risk  management  activities 
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Taking  Action  to  Enhance  Performance 
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Trade  Between  Advanced 
Capability  or  Increased  Maturity 
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System  Maturity 
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•  Areas  of  product  risk  to  end 
SoS  performance  become 
identifiable. 

•  Impact  of  technology  trade 
options  quickly  understandable 


System  Increment  1 
System  Increment  ? 
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Implementing  Effective  Requirements 
Management  &  the  Drive  Towards  Commonality 


Initial  Requirements  Structure 


LI- Governing 
Documents 


a 

a 

a 

a 

LCS  CONOPS 


12  Mission 

Package 

Documents 


13-  Mission 

Module 

Documents 


System 

Documents 


Mission  Package  focused.  Commonality  Grew  from  Requirement  Gaps 


Present  Requirements  Structure 


Commonality  starts  to  drive  the  MP  (present  and  future)  technologies 


•  Establishment  of  a  Cross  Organization  Requirements  Database 

•  Enforced  CM  process  involving  all  SE  stakeholders 

•  Implementation  of  a  Cultural  Change  towards  Common  Solutions 
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Going  Beyond  EVMS-Electronic  Task: 
Management  &  Tracking 
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Program  Management  tool  providing  the  entire  program  management  team  visibility 
into  and  the  ability  to  track  funding  and  tasking  allocations  by  year,  color,  type,  and 
organization  from  start  of  task  definition  through  funding  approval 
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Going  Beyond  EVMS-Electronic  Task: 
Management  &  Tracking 


Translating  complex  management  data 
into  displayable  and  actionable  information 
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Understanding  and  Influencing  SoS  Reliability 


•From  Flight  0+  CDD  June  2009,  Focused  Mission  Package  Annexes:  Required  A„ ; 


(T)  CCJOKey 
Characteristics 

(U)  Key  Performance 
Parameter 

(U)  Development 
Threshold 

(Ft  Development 
Objective 

(l')  Resilient 
Persistent 

4.4.5(F)  ASW 

Mission  Package 
Materiel  Availability 

(O  0.6-1 

(F)0."U 

CDD 

Requirement 

Target  \ 

Target  A0 

Target  A,Q 

Threshold  .64 

0.887 

0.738 

0.655 

Objective  .712 

0.887 

0.809 

0.718 

Inactive 


In  Repair 
&Test 


In  Transit 


Tech  Directive/ECP 
Modification 


In  Storage 


In  Transit 


Deployed 

Unavailable 


Deployed 


Deployed 
Operational 
12  :% 


Simulation 


Ave  =  6,  Std.  Dev.  =  2 


fgr  IVni 


Am  =  Aa  *  Ao 


Ave  =  3,  Std.  Dev.  =  3 
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Understanding  and  Influencing  SoS  Reliability 


Mission  Function  1  Component  String 


Mission  Function  2  Component  String 


Operational  strings  are  analyzed  to  identify  the  components  required  to  execute  independent  mission 
functions  of  the  system 

An  assessment  of  the  string  to  achieve  a  Mission  A0  contribution  is  performed 

Common  components  (nodes)  which  form  a  critical  function  in  more  than  one  mission  function  are  identified, 
operational  time  is  calculated  for  each  mission  it  touches  over  the  deployment  cycle 

Allocation  of  the  Mission  A0  decomposes  to  an  A0  requirement  at  the  component  and  Lowest  Replaceable 
Unit  (LRU)  level 
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Future  Steps:  Evaluating  Technology 

Analysis  and  Insertion 


1.  Develop  System  Architectures 


System  architecture  provides  the  loundation  lor 
system  maturity  assessments  (SRL,  CRL,  etc.) 


2.  Populate  Architectures 


Details  of  the  system  architectures  and  the 
IcchnoioRics  are  stored  in  the  Arch  tecturc 


3.  Define  Trade-Off  Analysis  Criteria 


'Customize  Applicable  Trade-Off  Criteria 


Cngin«#r 


'Review  Proposed  Trade-Off  Criteria 


Sv*t««m 

Engine*! 

IPT 


'Approve  Analysis  Criteria 
PM 


4.  Define  Analysis 
Criteria  Weightings 


We  gnting  values  are 
selected  based  on  the 
programs  current  focus 
and  direction 


5.  Calculate  Trade-Off  Analysis 

Architecture  Tool 


B0E10-0 

Data  of  the  current  y  utilized  and  available  technolog  es 
arecoinparedinthe  analysis 

Results  a'e  displayed  in  ranked  order,  with  snown  issues 
foreach  technologies  dispayed 


6.  Interpret  &  Apply  Calculated  Results 
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Outputs  of  the  analysis  are  used  to  make  decisions 
that  will  Impact  the  future  Iterations  (and  technology 
refreshes)  of  the  Mission  Packages 
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Future  Steps:  Performance  Level 

Assessment 


1.  Map  Technologies  to  KPPs  and  Maturity 


Goal:  Predict  the  ability  of  a  complex  systems  to  achieve  required  perforttidtrce 


1.  Map  the  Systems  to  their  impacts 
or>  key  performance  parameters 


2.  Map  the  maturity  development  ot  the 
Systems  to  Che  sos  development  schedule 


«UiA«l  MaSatrfy 


3.  Develop  a  relationship  between  system  usage  satisfying  a  KPP  in  a  SoS  and 
rts  maturity  (In  terms  of  a  vrelghted  value)  against  anticipated  performance 


•  Low  cost,  computation  simple  methodology  for 
rapidly  gaining  insight  into  SoS  Performance 

•PLM  may  provide  insight  into  incremental 
capability  compared  with  performance 
requirements. 

•  Evaluation  tool  for  new  capabilities  prior  to  their 
being  incorporated 


2.  Implement  impact  of  SoS  CONOPS  usage  options 


4.  Adjust  for  usage  impact  under 
various  employment  options 


conopsa„  -  PT..  *  rT  .. 
CONOPSst  ■  oT.,  *  tTj,  • 
CONOPS..  -6T,.  -  pT,. 


rT:. 


6.  Use  predictions  of  improved 
maturity  (SRL)  over  time  to 
denve  a  predicted  growth  path  of 
performance  for  SoS 


Average  the  results  from 
individual  employment  options  to 
obtain  insight  into  ability  to 
achieve  oOtarnment  of  the 
desired  performance  parameter 


KPPmakcm  -  ICON’OPa.  CONOPb.  CONOPt  |-AVOtCONOPA^CONOPn-CONOPr) 


3.  Conduct  Variance  Analysis 


7.  Use  estimates  of  performance  ane  maturity  to 
define  predictions  of  performance 

8.  Use  variances  of  the  usage  rates 
to  establish  bands  of 
performance  based  on  varying 
^  usage  options  of  the  individual 

_ ~  systerns/modules 


9.  As  data  is  gathered,  updated  predictions/ 
calculations  to  verify  if  development  is 
proctwdmy  «*»  <ta»i'*d 
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Concluding  Thoughts 


•  System  of  Systems  (SoS)  implementation  is  an 
exceptional  integration  challenge. 

>  Managing  multiple  interfaces  well  and  constantly  is 
key. 


•  Effective  employment  of  modular  MPs  is  feasible 
with: 

>  Modularity,  open  business  model,  and  open  systems 
architecture  supported  by  financial  and  programmatic 
tools  that  address  complexity. 


•  PMS  420  is  at  the  forefront  of  SoS  acquisition. 

>  New  methodologies  are  being  developed  to  provide 
technical  and  program  management  insight 
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QUESTIONS? 
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